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Abstract 

Our paper describes the design and development of a set of online 
professional development modules to support academic staff in improving 
and enhancing their teaching. We show how we created a learning design to 
allow staff to quickly and easily develop their teaching in line with University 
performance expectations whilst also providing staff with a sound basis for 
documenting their professional development using a newly created ePortfolio 
tool. We present data from the evaluation of the modules and discuss a 
research project for evaluating the ultimate impact of this initiative on 
student learning. 
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Introduction 

Our paper describes the design, development and delivery of a set of professional development 
modules to support academic staff within a Faculty of Medical and Health Sciences in developing 
their teaching. The modules were created in the context of a broader project to provide staff with a 
range of teaching supports and resources in line with the Faculty’s strategic aim of achieving 
excellence in teaching. The additional resources included a teaching performance rubric to guide 
staff in assessing their teaching performance and an ePortfolio to allow academics to maintain 
teaching records for formative and summative purposes. 

From a strategic point of view the performance rubric and the ePortfolio were both considered 
necessary in order to motivate staff to engage with the professional development modules. From a 
learning design point of view, the challenge was to develop professional development modules 
that would allow staff to quickly and easily develop their teaching in line with University 
performance expectations whilst also providing staff with the means to document their 
professional development for formative and summative purposes. We will show how we achieved 
these design aims through underpinning the learning design for the modules with a model of 
scholarly teaching. 

Context 

Continuing Professional Development (CPD) consists of formal and/or informal learning that 
leads to the enhancement of knowledge, skills and personal attributes necessary to carry out 
professional duties (Gosha, Billionniere, Gilbert, & Ramsey, 2010; Guskey, 2000; Stefani, 2005). 
In this paper, we will refer to CPD for teaching as Educational Professional Development (EPD) 
(Knight, 2006). EPD is important for the renewal and vitality of the organization the professional 
development of individual staff members and for improving teaching to enhance and improve 
student learning (Guskey, 2000; Lanthan, Camblin, & Steger, 2000). However, longitudinal 
research into the benefits of EPD has been lacking (Dede, Jass Ketelhut, Whitehouse, Breit, & 
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McCloskey, 2009; Mouza, 2009) and this is something that the Learning Technology Unit at the 
Faculty of Medical and Health Sciences (http://www.fmhs.auckland.ac.nz/faculty/ltu/) has been 
addressing in the context of other EPD initiatives (Doherty, 2010; Doherty, Blake, & Cooper, 
2009; Doherty & Cooper, 2009). We will return to the question of the evaluation of EPD 
initiatives at the end of our paper. 

A range of EPD opportunities are available for academic staff at the University of Auckland. All 
new teaching staff must take a three-day course - provided by the University’s Centre for 
Academic Development (CAD) - on aspects of teaching at the University. This course is designed 
to introduce early-career academics to key concepts and strategies for university teaching and 
learning and to provide a useful refresher course for more experienced teachers. The CAD also 
offers a Certificate in Academic Practice. This is a credit-bearing program that provides university 
lecturers with a structured, research-based educational environment in which they can explore 
theories of tertiary teaching and academic citizenship. Finally, the CAD provides a variety of 
workshops covering all aspects of university teaching & supervision including teaching to large 
classes, effective tutoring, effective assessment practices, use of information technologies in 
teaching and good practice in postgraduate supervision. 

We designed our EPD modules to expand on existing University EPD offerings in two ways. First, 
we wanted to provide EPD modules with content designed to meet the particular needs of medical 
and health science educators. For example, clinical educators have to assess student performance 
in hospital setting and primary health care settings. This requirement is specific to a medical and 
health sciences educational setting. Secondly, we knew from informal conversations and from an 
unpublished FMHS staff survey that many staff - and in particular clinical educators - found it 
difficult to find the time to attend face-to-face workshops. With this in mind we wanted to provide 
online EPD content that would allow educators to engage in EPD at a time and place of their own 
choosing. 

Our strategy for engaging educators with the EPD modules focused on making a connection with 
the university promotion processes. In order to realize this strategy we established a project team 
that included: the Director of the Learning Technology Unit (LTU) in the role of project leader; 
the Associate Dean Education; the Director of the Centre for Medical and Health Sciences 
Education (CMHSE); and academic staff members from the LTU and CMHSE. The project 
deliverables - tangible products resulting from the project - included a teaching performance 
rubric, the EPD modules and an ePortfolio. The purpose of this paper is to explain the EPD 
module design process including the integration of the performance rubric and the ePortfolio into 
the EPD modules. We will, therefore, provide details of the performance rubric in the first 
instance. We will then describe the design and development of the modules. Finally, we will show 
how the ePortfolio was integrated into the module learning design. 

Performance rubric 

The University of Auckland policy document on academic grades, standards and performance 
criteria sets out a variety of activities undertaken in the three scholarly domains - teaching, 
research and service - and provides examples of the sorts of achievements associated with the 
various performance levels (satisfactory, merit, excellence and distinction). The document is used 
to aid individual academics in career and promotion planning, to aid academic managers in 
providing appropriate and consistent advice to potential promotion applicants and to aid staffing 
committees in making recommendations about promotion. Consideration for promotion to any 
grade involves two steps. First, the committee must be satisfied that the candidate has reached a 
satisfactory (or in the case of Associate Professor - merit) level of performance consistent with his 
or her grade of appointment in each of the three broad areas for promotion. Second, in order to be 
promoted, the candidate must also demonstrate merit (for promotion to Senior Lecturer), 
excellence (for promotion to Senior Lecturer over the bar), and distinction (for promotion to 
Associate Professor), in at least one of the three broad areas. 
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We wanted to provide staff with a straightforward way to identify the University’s teaching 
performance expectations and so we decided to create a Faculty representation of the University’s 
teaching performance areas and performance indicators in the form of a teaching performance 
rubric. An example from the teaching rubric (Table 1 below) will help to make clear how the 
performance rubric works in practice. Table 1 presents the rubric for one teaching performance 
area, ‘Delivery of teaching to facilitate learning’. Examples of the different sorts of activities that 
might be evidenced are listed under the four performance levels. We can see, for example, that a 
satisfactory performance might be evidenced through making use of a range of teaching and 
learning methodologies. Interested readers can access the full performance rubric at 
http://www.fmhshub.auckland.ac.nz. 

Table 1: Teaching performance areas and associated performance activities 



Satisfactory 

Merit 

Excellence 

Distinction 

Delivery 

Competent use of 

Innovation in 

Promotes effective 

National / 

of 

a range of teaching 

teaching 

teaching practice at 

international 

teaching 

and learning 

methodologies 

institutional level 

standing in 

to 

methodologies to 

and evaluation of 

through mentoring. 

furthering 

facilitate 

engage students in 

impact on 

peer review, 

understanding of 

learning 

the learning 

learning. 

workshops or 

and improving of 


process 

Applies 
appropriate 
pedagogical 
frameworks to 
the improvement 
of own teaching 
practice 

seminars. Researches 
into approaches to 
teaching that improve 
learning outcomes. 

teaching and 
learning. 


The rubric is not a checklist that dictates that a certain number of specific activities must be 
demonstrated. Rather the rubric acts as a guide that outlines the kinds of activity that would be 
regarded as demonstrating performance at a certain level. There is, therefore, room for 
interpretation when providing examples of particular activities for the various performance levels. 
For example, staff might demonstrate innovation by introducing and evaluating a new assessment 
method such as a peer review assessment practice. 

Educational professional development modules 

Scholarly teaching 

Once we had produced the teaching performance rubric, we faced two challenges. First, we needed 
to create a learning design that would connect the EPD module content with the teaching 
performance rubric. Secondly, we needed to engage staff in a learning process that would lead to 
evidence based teaching improvement that might be documented in the ePortfolio for formative 
and summative purposes. We were also guided by another consideration. We wanted to engage 
educators in a process of scholarly teaching because developing the scholarship of teaching can 
help to raise the status of teaching relative to discipline research (Boyer, 1990; Healey, 2008; 
Schroeder, 2007; Trig well & Shale, 2004) and because there is some evidence that scholarly 
teaching provides a solid foundation for developing teaching to improve student learning (Brew & 
Ginns, 2008; Spronken-Smith & Kingham, 2009; Tsang, 2010). The scholarly model of teaching 
eventually provided us with a solution to the first two design challenges - how to connect the 
modules with the rubric and the ePortfolio - and we will, therefore, describe the scholarly teaching 
model in the first instance. 

Richlin (Richlin, 2001) provides the model of scholarly teaching ( Figure 1 below) that we used in 
the first stage of our design process. The scholarly teaching process can essentially be divided into 
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three distinct steps. Step one involves identifying a teaching a learning challenge and making an 
informed decision regarding the solution to the challenge. Step two involves implementing a 
change in teaching practice and recording what happened as a result of the change. Step three 
consists of evaluating the impact of the intervention. Scholarly teaching is distinguished from the 
scholarship of teaching in terms of the requirement that scholars of teaching disseminate research 
findings through peer-reviewed publications. We made the deliberate choice to follow the model 
of scholarly teaching because the University of Auckland is a research-intensive university with 
many academics lacking the time and /or inclination to engage in the scholarship of teaching. This 
situation is not unusual in tertiary institutes of education (Gossman, Haigh, & Jiao, 2009) and is 
particularly pronounced in research intensive universities (Asmar, 2002a, 2002b, 2004). 


Observe 

Teaching ><Learning 
Connection™ 
problem or 
opportunity 

and document Consult 

the baseline literature 


i 


Scholarly Teaching 
Check results against baseline •* 


Choose and Conduct Document Analyze 

apply an systematic observations results 

intervention observation 


Obtain 
* peer 
evaluation 


Knowledge Base of 
Teaching and Learning 
In Higher Education 

Add to 

I I 

Disseminate,Submit Prepare Place into Synthesize Identify 

publish, for manuscript context of Results key 

present peer review knowledge base issues 

The Scholarship of Teaching 


Figure 1: Scholarly teaching 


After some consideration we saw that the scholarly teaching steps could be embodied in the EPD 
learning process in a very straightforward manner. Essentially, educators are guided through step 
one - identifying a challenge and making an informed choice about a solution - through engaging 
them with the rubric and the modules. The reason for this is that the rubric specifies performance 
areas and performance indicators (the challenge) whilst the module content provides research 
informed solutions to the challenge. Step two - implementation, analysis and documentation - is 
realized through introducing educators to the ePortfolio within the modules and by directing them 
to record and analyze the results of their implementation for formative and summative purposes. 
Finally, step three - evaluating the impact of the change with reference to the original challenge - 
can be achieved through requiring educators to describe what they need to do (next steps) as a 
result of having implemented a change in their teaching practice. 


Module design and development 

Having mapped out our learning design at a broadly conceptual level, we set about creating a more 
detailed design for the EPD modules. For our purposes EPD extends to the full range of teaching 
and teaching related activities to be found in a Faculty of Medical and Health Sciences including: 
delivering lectures; running tutorials; teaching clinical skills; teaching medical and health science 
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interns in a clinical setting (e.g. hospital or doctor’s surgery); course design; curriculum design; 
student supervision and pastoral care of students. With the exception of the clinical teaching 
components, these teaching activities will be common across all institutes of higher education and 
the content of this paper should, therefore, be relevant to educators in a variety of settings. 

We wanted to create a learning design that would engage educators in progressively more 
demanding levels of cognitive activity appropriate for scholarly teaching. Therefore, we took 
Anderson and Krathwohl’s (Anderson & Krathwohl, 2001) revision of Bloom’s taxonomy 
(Bloom, 1956) of the cognitive domain in order to represent the different levels of possible 
learning. Readers may notice that the taxonomy represented in Table 2, differs from the standard 
representation of Anderson and Krathwohl’s taxonomy. In the usual representation the order is as 
follows: knowledge; comprehension; application; analysis; synthesis (create / build); and 
evaluation. In our representation analysis and synthesis (designing a teaching intervention) precede 
application and evaluation (teaching practice and ePortfolio records). The reason for our revised 
representation is that we wanted an order for the levels of the cognitive domain in which the 
learners - in this case, teachers - engage in progressively deeper levels of learning mapped to the 
scholarly teaching model already outlined. 

Table 2: The act of learning 


The Act of Learning 

Bloom’s Taxonomy 

Acquisition [New knowledge] 

Transformation [Application of 
knowledge] 

Evaluation [Determining utility] 

Knowledge [Recalling facts] 

Comprehension [Constructing new meaning] 
Analysis [Subdividing and contrasting) 

Synthesis [Reorganizing elements] 

Application [Relating learning to new situation] 
Evaluation [Determining potential utility] 


Based on the initial design work outlined above, we wrote a series of intended learning outcomes 
for the EPD modules. The outcomes addressed each of the teaching performance areas found in 
the teaching performance rubric. We then structured the module content to scaffold educators 
(Scardamalia, et al., 1992; Scardamalia, Bereiter, Swallow, & Woodruff, 1989; Vygotsky, 1978) 
through a series of reflective activities that culminated in asking teachers to start an ePortfolio 
record to baseline the teaching challenge/opportunity (Richlin, 2001) that they had identified as a 
result of their engagement with the EPD modules. This represents the last learning step in the 
module and the first step in creating an ePortfolio record. We then used a custom course creation 
tool to create basic content for the first two modules. The content consisted of text, images, and 
links to additional research resources (journal articles and websites) for those educators wanting to 
take their learning further. 


Development 

We showed the two draft modules to 6 educators - from the level of tutor through to professor - 
and asked for feedback in the following areas: buy in/use of the teaching modules by FMHS 
academic staff and clinical teachers; design and layout of the modules; important/desirable 
content. We received the unanimous response that academics did not like the modular approach 
that we had taken. This response from one of the interviewees captures the sentiment. 

Don’t give staff modules - don’t make it feel like hard work. Tell us what can be done. 

One respondent indicated that it would be most useful to be able to access information based on 
specific questions that teachers ask when they teach. Based on an indication in the literature that 
professional learning tends to be informal and practice based (Knight, 2006), we picked up on this 
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suggestion and decided to present the content in terms of a series of questions and related answers 
categorized under the different teaching performance areas in the teaching and learning rubric. 
This strategy provided for just in time online learning (Gosha, et al., 2010) connected with what 
academics do in their teaching. The module structure can be seen in the navigation menu on the 
left hand side of Figure 2 below where one of the modules - ‘Delivery of teaching to facilitate 
learning’ - has been expanded to reveal the questions. The questions relate both to teaching 
practice and to the sorts of activities that might be used to evidence teaching performance. For 
example the question, ‘Flow can I engage my students in learning?’ is the sort of question that 
might be asked in teaching whilst the answer to the question provides educators with the 
information that they need to evidence satisfactory performance in the ‘Delivery of teaching to 
facilitate learning.’ 


Home 

Delivery of teaching to 
facilitate learning 

What makes a good 
teacher? 

How do I plan my 
teaching? 

How can I engage my 
students in learning? 

What is the best 
teaching approach? 

When should I use 
technologies? 

How do I evaluate the 
impact of an 
innovation? 

Design and planning of 
courses/programmes 

Assessing learning & 
providing feedback 

Addressing diversity 

Evaluation of practice 
and continuing 
professional 
development 

Contribution of 
scholarship, research 
& professional 
activities to teaching & 
learning 

Student Support, 
Guidance and 
Supervision 


GLOSSARY OVERVIEW PERFORMANCE myEPORTFOLIO EXEMPLARS SUPPORT RESOURCES GLOSSARY FEEDBACK 

Delivery of teaching to facilitate learning 


Teaching Activity Area and Performance Criteria 

This activity area covers the delivery of teaching to facilitate learning. Teaching might be delivered through 
lectures, tutorials, seminars, laboratory work, workplace-based teaching and through distance/flexible teaching 
involving the use of technologies. 

When completing your annual performance review, your continuation application or your promotion application 
you can make use of the criteria below to guide you in selecting specific examples of activities or techniques 
that you have used to deliver teaching to facilitate learning. For example, you might want to specify the 
teaching and learning methodologies that you have used to facilitate learning or you might want to describe an 
innovation in your teaching such as the use of a particular technology. If you have made use of 
myEPORTFOLIO, you will already have recorded this information. 

You can view the full range of activity areas and performance criteria by visiting this web page. 


The promotion criteria for this activity area are: 


Satisfactory 

Merit 

Excellence 

Distinction 

Competent use of a 

Innovation in teaching 

Promotes effective 

National / international 

range of teaching and 

methodologies and 

teaching practice at 

standing in furthering 

learning methodologies 

evaluation of impact on 

institutional level 

understanding of and 

to engage students in 

learning. 

through mentoring peer 

improving of teaching 

the learning process. 

Applies appropriate 
pedagogical frameworks 
to the improvement of 
own teaching practice. 

review, workshops or 

seminars. 

Researches into 
approaches to teaching 
that improve learning 
outcomes. 

and learning. 


Figure 2: Teaching performance area and performance activities (screen capture) 


We also removed the learning outcomes from the modules because these defined the EPD modules 
as a course that had to be worked through. Rather than creating content / learning activities around 
learning outcomes, we structured each module to scaffold learners through a series of reflective 
tasks that involved progressively deeper levels of learning culminating in learners designing 
(synthesis in the learning taxonomy) a teaching intervention that would meet their identified 
teaching challenge. For example, in the section on ‘What makes a good teacher?’ educators are 
presented with some basic information on good teaching. This is followed by three videos of well- 
known university educators talking about what makes a good teacher. Users are then asked to 
complete a teaching perspective inventory to give them insight into the way in which they 
approach their teaching. This is followed by information on a teaching philosophy. Users are then 
asked to write their own teaching philosophy. Finally, we suggest that users start an ePortfolio 
record and that they write their own teaching philosophy. The learning activities in this - and other 
modules - reflects EPD research suggesting that EPD activities must engage learners in real world 
learning that has direct relevance to their teaching (Bell & Morris, 2009; Mouza, 2009; 
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Samarawickrema, Benson, & Brack, 2010; Stefani, 2002). The learning structure also progresses 
users through the levels of the cognitive domain outlined above. 

Interviewees made particular mention of wanting to know more about teaching with technologies. 
Given the university’s commitment to enhancing teaching through the use of technologies, we 
included a specific section on technology use under the performance area of ‘Delivery of teaching 
to facilitate learning’. Interviewees also suggested the use of short informative video clips of well- 
known educators within the Faculty / University. We recorded FMHS and University teaching and 
learning “personalities” talking about particular aspects of teaching and learning. The videos that 
we produced included various educators talking about what makes a good teacher (see above), the 
winner of the Faculty’s 2009 award for PhD supervision talking about excellence in supervision 
and recipients of various Faculty and University teaching awards talking about teaching activities 
such as delivering teaching to facilitate learning and engaging students in learning. Finally, 
interviewees also requested best practice examples. With this suggestion in mind we created an 
exemplar page with best practice teaching examples from across the Faculty. 


Evaluation 

We asked 5 (N=5) academics to review the beta version of the modules. Their positions in the 
Faculty were as follows: Senior Tutor (2); Lecturer (1); Associate Professor (2). The broad 
purpose of the evaluation was to gather feedback on the content of the site and to assess site design 
/ usability. 

We asked, “How well do you think the web site meets the teaching and learning development 
needs of the Faculty?” and participants could respond on a five point Likert scale that ranged from 
“Very Well” to “Not Well At All.” 4 (n=4) participants responded, “Very Well” and 1 (n=l) 
participant responded with “Well.” Participants were also given the opportunity to explain their 
responses. These responses are representative. 

Content is linked explicitly to University and faculty performance criteria. This could be 
stifling, but it isn't. The material is well summarized, clear and practical. The 'talking heads' 
videos are, in the main, engaging and complementary. 

It looks comprehensive and very user friendly. It also provides easy access to relevant 
resources. I particularly like the three progression steps e.g. taking if further & link to e- 
portfolio. 

We asked, “Will you make use of this website for your professional development?” and 
respondents could answer, “Yes”, “No” or “Maybe.” 4 (n=4) participants responded with “Yes” 
and 1 (n=l) participant responded with, “Maybe.” The respondent who answered “Maybe” was a 
senior member of Faculty who qualified the response by explaining that the content of the site was 
better suited to those with less teaching experience. Additional comments on whether they would 
make use of the site for their own professional development included: 

Most likely I'd recommend it to other staff starting out in teaching or staff needing to 
generate information for promotion. 

Because it refers explicitly to promotion criteria. 

We asked, “How valuable/useful do you think this website is for supporting you in your own 
professional development?” and participants could respond on a five point Likert scale from “Very 
Useful” to “Not Useful At All.” 2 (n=2) participants responded with, “Very Useful”, 2 (n=2) with 
“Useful” and 1 (n=l) with “Quite Useful.” The respondent who answered “Quite Useful” was the 
same senior member of Faculty who felt that the content was better suited to those with less 
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teaching experience. Comments from academics on the perceived usefulness of the site for their 
own professional development included: 

The level is well pitched. It would allow a novice teacher to get started and provides 
something to stretch more experienced teachers. 

This is perfect timing, as I need to apply for a promotion this year. As my role in the 
University does not have a research commitment I often think that it is difficult to reflect 
my value and contribution to this organization but this website provides good scaffolding to 
let me highlight strengths and innovation. 

The comment concerning timing and promotion came from a Senior Tutor, a position that does not 
require research on the part of the appointee. Teaching performance is, therefore, very important in 
a promotion application. 

Finally, we asked “Would you recommend this website to other educators?” and participants could 
respond with “Yes”, “No” or “Maybe.” All 5 (n=5) participants responded with “Yes”. There was 
only 1 comment for this question. 

In process of employing new tutor/lecturer and will definitely point them in this direction. 

We asked, “How visually appealing is the web site?” and participants could respond on five point 
Likert scale from “Very Appealing” to “Not Appealing At All.” 3 (n=3) participants responded 
with “Very Appealing” and 2 (n=2) participants responded with “Appealing.” 

Comments on visual appeal were wholly positive with this comment being representative. 

The layout is attractive without being dominant. Some sites are great to look at and 
navigate, but underwhelming in content and application. The design isn't 'fussy'. I like the 
tab structure and the uniformity of approach. 

We asked, “How easy/difficult is it to navigate the website?” and participants could respond on 
five point Likert scale from “Very Easy” to “Very Difficult.” 4 (n=4) participants responded with 
“Very Easy” and 1 (n=l) participant responded with “Easy.” There was only one comment for this 
question. 

Smooth links to readings and external sites (I found only one broken link - teaching 
philosophy) 

There were no negative comments about site content or site design and as a result of the 
evaluations we judged that we had produced a resource that was easy to use with appropriate 
content for the EPD needs of the Faculty. However, we would note that comments from the more 
senior academic suggest that the content may be more appropriate for “novice” teachers. This 
judgment is further supported by the comment that the site would be useful for new staff. The fact 
that the EPD modules appear to be more suitable for new / less experienced teaching staff is not a 
particular concern because we had made the decision to focus the EPD activities on activities 
related to the first two performance levels (satisfactory and merit). The reason for making this 
decision was that the first two performance levels address teaching competencies necessary for 
excellent teaching. The remaining performance levels (excellence and distinction) address 
competencies that extend and build upon the core competencies and we took the view that 
competencies for these levels could be developed both informally through peer support and in 
other professional development contexts such management and leadership courses. For example, 
excellence in teaching and learning requires leadership in education at a school / University level 
and distinction in teaching requires leadership in education at an international level. 
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ePortfolio 

From a learning design perspective, the ePortfolio is important because it provides academics with 
a means to record their EPD activities along with the results of their teaching interventions. A 
portfolio is a collection of information about teaching practice (Tucker, Stronge, Gareis, & Beers, 
2003). The benefits of maintaining a teaching portfolio include the opportunity to reflect on 
teaching practice, taking ownership of the need for professional development and having a formal 
and verifiable record of teaching practice. (Stefani & Diener, 2005; Tucker, et ah, 2003). 

Portfolios can be maintained for formative and/or summative purposes. At a formative level the 
portfolio functions as a vehicle for a teacher to maintain an ongoing reflective record of how they 
are doing in their teaching. At a summative level, a portfolio can be used to evidence teaching 
performance during reviews, when applying for promotion and when applying for teaching grants 
or awards for excellence in teaching (Seldin & Miller, 2009). Teaching portfolios can be 
maintained as a hard copy or as in an electronic format (ePortfolio). The ePortfolio is perceived to 
offer distinct practical benefits compared with hard copy portfolios. For example, ePortfolios 
provide a way for educators to document their teaching and to manage voluminous amounts of 
portfolio evidence whilst also providing a standardized presentation format for the various 
reviewers. Online ePortfolios are readily available anywhere and at any time as long as the user 
has an Internet connection. (Swan, 2009). 

We are aware that the use of teaching portfolios is not innovative. Indeed our early research for 
this project examined the use of teaching portfolios at five Universitas 21 universities: University 
of Edinburgh; University of Melbourne; University of Queensland; University of British 
Columbia; and the University of New South Wales. Additionally, there is voluminous literature on 
teaching portfolios from the United States where teaching portfolios have been recommended for 
use by in-service teachers, by pre-service teachers and for use by educators in institutes of higher 
education (Glassick, Huber, & Maeroff, 1997; Seldin, 1997; Soulier & Recker, 2001; Swan, 2009; 
Tucker, et ah, 2003). Our research and literature review revealed a number of ePortfolio options 
ranging from use of off-the-shelf word processing software such as Microsoft Word to create 
portfolio templates through to custom built ePortfolio systems to provide a feature rich online 
ePortfolio system. 

Having surveyed the options and considered our model of scholarly teaching, we decided to create 
a customized ePortfolio system that would allow academics to quickly and easily create teaching 
records aligned with taking a scholarly approach to teaching. We opted for a custom system 
because we wanted to offer a straightforward online system with additional functionality including 
a search feature and an upload function for storing associated evidence. Developers at the 
University’s Centre for Academic Development developed the custom ePortfolio system. We 
consulted with the Faculty Staffing Committee - the committee responsible for making 
recommendations concerning promotion - and with individual academics during the design 
process. However, time pressure meant that no formal evaluation was carried out before the 
release of the ePortfolio. This meant that we had a “soft launch” of the ePortfolio during which we 
asked academics to use the system over a two-month period and to report back to us on the 
usefulness of the system. 

The form fields (Figure 3 below) for the ePortfolio include: a record description for providing an 
overview of the portfolio entry (teaching and learning connection); a reflection section for entering 
details concerning the nature of the teaching challenge / opportunity (baseline activity); an action 
field for describing what was done (research-based intervention); an evaluation field to present the 
results of the intervention (analysis of results); and a next steps field for describing follow on 
actions from the intervention (check results against baseline). The ePortfolio therefore provides a 
way to record each step in the scholarly teaching process whilst also providing a means for 
academics to create teaching records suitable for summative purposes including applying for a 
promotion. Users can also upload evidence of their teaching practice including, for example, 
electronic student evaluations. The completed record must be printed and appended to university 
review documentation. This choice to provide a print function to allow academics to generate their 


2010 Vol. 4 No. 1 


9 








Journal of Learning Design 


Iain Doherty 


teaching records was made because we did not have the resources to develop a more complex 
system that would automatically integrate the teaching records with the various university forms 
used during formal review processes. Additionally, informal discussions with academics had 
indicated that academics wanted a very straightforward system. 


Record description (How can I engage my students in learning?) 



Support evidence [optional] 

Upload file Choose File ) no file selected 

(Save ) ( Cancel) 


Figure 3: ePortfolio form fields 
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Evaluation 

Research to measure the impact of this project might be carried out on a number of different levels 
including: the usefulness of the modules for teaching staff; the usefulness of the teaching and 
learning rubric for teaching staff, academic managers and members of the staffing committees; and 
the usefulness of the ePortfolio for teaching staff, academic managers and the members of the 
staffing committees. Each of these evaluation levels is important from a strategic point of view, 
particularly in terms of determining the degree of systemic change within the Faculty. However, 
the core purpose of EPD is to make teaching both more rewarding and more effective (Lanthan, et 
al., 2000) and systemic change per se means very little unless teaching does actually become more 
rewarding and more effective. 

At the time of writing we are developing a research project to evaluate the impact of this project at 
a systemic level. We also want to evaluate teacher development and impact on student learning. 
With the question of systemic change in mind, we will be evaluating: use and perceived value of 
the teaching modules (all teaching staff); use and perceived value of the electronic teaching 
portfolio (teaching staff, academic managers and members of the staffing committee); and use and 
perceived value of the teaching guidelines (teaching staff, academic managers and members of the 
staffing committee). We are still in the process of creating the questionnaires. However, we can 
give some indication of the evaluation specifics. The EPD module questionnaire will ask questions 
about the user experience of the EPD modules - navigation, design, structure and clarity of 
presentation - and about degree of engagement with the EPD content. For example, we are 
intending to ask about the extent to which teachers engaged with the research base for the 
modules, the extent of independent research into teaching practice and about any changes that 
teachers made to their teaching as a result of engaging with the modules. The performance 
framework questionnaire will ask the three groups about the extent to which they made use of the 
framework and about the perceived value of the framework in their particular contexts. For 
example, we will ask the Staffing Committee whether they found the framework useful when 
making recommendations about promotion applications. The ePortfolio questionnaire will ask the 
three groups about the extent to which they made use of the ePortfolio and about the perceived 
value of the ePortfolio in their particular contexts. For example, we will ask academic managers 
whether they judged any teaching portfolios prior to a staff member making an application for 
promotion. 

Thomas Guskey (Guskey, 2000) draws on a number of evaluation models - including the well 
known Kirkpatrick model - to describe four levels of evaluation that are important if we want to 
measure whether teaching has become more rewarding and more effective as a result of EPD. 
These levels are: participants’ reactions to the EPD; whether or not participants achieved the 
desired learning outcomes; participants’ use of new knowledge and skills in their teaching; and 
resultant improvements in student learning. Our current evaluation plan focuses on participants’ 
use of new knowledge and skills (the EPD questionnaire) and on resultant improvements in 
student learning (the ePortfolio questionnaire). For example, the EPD questionnaire asks about 
changes teachers made to teaching and about their evaluation of the changes that they made. We 
have more work to do to develop a robust evaluation of the EPD modules. For example, Guskey 
(Guskey, 2000) makes the point that it is important to evaluate the user experience along with 
whether or not participants achieved the desired learning outcomes because a positive experience 
along with achieving learning outcomes are both necessary conditions for making effective 
changes to teaching practice. The work of Reeves and Hedberg on evaluating interactive learning 
systems may also prove useful to us (Reeves & Hedberg, 2003) as we develop our plan for 
evaluating the EPD modules and we are looking at online evaluation resources provided by Reeves 
(http://www.evaluateitnow.com/). Finally, we have more work to do to complete the project 
evaluation plan as a whole and we are currently looking at evaluation resources from the 
Australian Teaching and Learning Council (ALTC) (http://www.altc.edu.au/extras/altc- 
gsep/index.html) in order to develop our evaluation plan. Work in this area will be completed by 
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late September 2010 at which time we will submit an ethics application to the University ethics 
committee. 


Conclusions 

Our paper has described the design and development of a set of EPD modules in the context of a 
project to achieve systemic change in the teaching culture of a Faculty of Medical and Health 
Sciences. We have explained the basis of our learning design and made particular reference to the 
model of scholarly teaching that underpinned our design decisions. We have presented evaluation 
data for the modules along with data reporting participants’ reactions to the pre-release version of 
the EPD modules. We have also shown how the ePortfolio played an integral part in the design 
and development of the EPD modules through providing the means for teachers to record their 
professional development for formative and summative purposes. Finally, we discussed our 
evaluation plan in some detail and indicated that we still have work to do in order to develop a 
robust evaluation framework. The EPD modules and the ePortfolio were launched in late April 
2010. The challenge now is to promote and embed these resources in the teaching culture of the 
Faculty. Work has already begun in this area and we hope to report on this process in 2011 / 2012. 
We will also be reporting on the results of our evaluations in 2011 / 2012. 
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